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This  paper  shows  an  example  of  how  simulated  evolution  can  be  applied  to  a  practical  optimization  problem,  and  more 
specifically,  how  the  addition  of  co-evolving  parasites  can  improve  the  procedure  by  preventing  the  system  from  sticking  at 
local  maxima.  Firstly  an  optimization  procedure  based  on  simulated  evolution  and  its  implementation  on  a  parallel  computer 
are  described.  Then  an  application  of  this  system  to  the  problem  of  generating  minimal  sorting  networks  is  described.  Finally 
it  is  shown  how  the  introduction  of  a  species  of  co-evolving  parasites  improves  the  efficiency  and  effectiveness  of  the 
procedure. 


1.  Introduction 

The  process  of  biological  evolution  by  natural 
selection  [5]  can  be  viewed  as  a  procedure  for 
finding  better  solutions  to  some  externally  im- 
posed problem  of  fitness.  Given  a  set  of  solutions 
(the  initial  population  of  individuals),  selection 
reduces  that  set  according  to  fitness,  so  that  solu- 
tions with  higher  fitness  are  over-represented.  A 
new  population  of  solutions  is  then  generated 
based  on  variations  (mutation)  and  combinations 
(recombination)  of  the  reduced  population.  Some- 
times the  new  population  will  contain  better  solu- 
tions than  the  original.  When  this  sequence  of 
evaluation,  selection,  and  recombination  is  re- 
peated many  times,  the  set  of  solutions  (the  popu- 
lation) will  generally  evolve  toward  greater  fitness. 

A  similar  sequence  of  steps  can  be  used  to 
produce  simulated  evolution  within  a  computer  [3, 
4.  12.  17-19].  In  simulated  evolution  the  set  of 
solutions  is  represented  by  data  structures  on  the 
computer  and  the  procedures  for  selection,  muta- 
tion, and  recombination  are  implemented  bv  algo- 
rithms that  manipulated  the  data  structures. 
Although  the  term  "simulated  evolution"  deliber- 
ately suggests  an  analogy  to  biological  evolution, 
it  is  understood  that  the  real  biological  processes 
are  far  more  complex  than  the  simulation:  simu- 


lated evolution  represents  only  an  idealization  of 
certain  aspects  of  a  biological  system.  Such  simu- 
lations are  sometimes  used  as  tools  for  under- 
standing biological  evolution  [15],  but  this  paper 
will  concentrate  on  the  use  of  simulated  evolution 
for  optimization;  that  is,  as  a  practical  method  of 
generating  better  solutions  to  problems.  Biological 
systems  will  serve  as  a  source  of  metaphor  and 
inspiration,  but  no  attempt  will  be  made  to  apply 
the  lessons  learned  to  biological  phenomena. 

As  an  optimization  procedure,  the  goal  of  simu- 
lated evolution  is  very  similar  to  that  of  other 
domain-independent  search  procedures  such  as 
generate  and  test,  gradient  descent,  and  simulated 
annealing  [13.  16].  Like  most  such  procedures, 
simulated  evolution  searches  for  a  good  solution, 
although  not  necessarily  the  optimal  one.  Whether 
or  not  it  will  find  a  good  solution  will  depend  on 
the  distribution  of  solutions  within  the  space. 

These  methods  are  all  useful  in  searching  solu- 
tion spaces  that  are  too  large  for  exhaustive  search. 
As  in  gradient  descent  and  simulated  annealing 
procedures,  simulated  evolution  depends  on  infor- 
mation gathered  in  exploring  some  regions  of  the 
solution  space  to  indicate  which  other  regions  of 
the  space  should  be  explored.  How  well  this  works 
obviously  depends  on  the  distribution  of  solutions 
in  the  space.  The  types  of  fitness  spaces  for  which 


■il67-27S9/9(l /S03.50  <  Elsevier  Science  Publishers  B.V. 
i  North-Holland) 


W.D.  Wilis  /  Co-evolving  parasites  improve  simulated  evolution 


229 


simulated  evolution  produces  good  results  are  not 
well  understood,  but  one  important  type  of  space 
for  which  it  works  is  a  space  that  is  independently 
a  good  domain  for  hill  climbing  in  each  dimen- 
sion. 

Another  attractive  property  of  simulated  evolu- 
tion is  that  it  can  be  implemented  very  naturally 
on  a  massively  parallel  computer.  During  the  se- 
lection step,  for  example,  the  fitness  function  can 
be  evaluated  for  every  member  of  the  population 
simultaneously.  The  same  is  true  for  mutation, 
recombination,  and  a  computation  of  statistics 
and  graphics  for  monitoring  the  progress  of  the 
svstem.  In  the  system  described  below,  we  rou- 
tinely simulate  the  evolution  of  populations  of  a 
million  individuals  over  tens  of  thousands  of  gen- 
erations. Since  these  simulations  take  place  on 
several  generations  per  second,  such  experiments 
take  only  a  few  hours. 

In  these  simulations,  individuals  are  represented 
within  the  computer's  memory  as  pairs  of  number 
strings  that  are  analogous  to  the  chromosome  pairs 
of  diploid  organisms.  The  population  evolves  in 
discrete  generations.  At  the  beginning  of  each 
generation  the  computer  begins  by  constructing  a 
phenotype  for  each  individual,  using  the  set  of 
number  strings  corresponding  to  an  individual 
(the  "genome")  as  a  specification.  The  function 
used  for  the  interpretation  is  dependent  upon  the 
experiment,  but  typically  a  fixed  region  within 
each  of  the  chromosomes  is  used  to  determine 
each  phenotypic  trait  of  the  individual.  Discrepan- 
cies between  the  two  bit  strings  of  the  pair  are 
resolved  according  to  some  specified  rule  of  domi- 
nance. This  is  similar  to  the  diploid  "genetic  algo- 
rithms" studied  by  Smith  and  Goldberg  [18]. 

To  simulate  selection,  the  phenotypes  are  scored 
according  to  a  set  of  fitness  criteria.  When  the 
system  is  being  used  to  solve  an  optimization 
problem,  the  traits  are  interpreted  as  solution  pa- 
rameters and  the  individuals  are  scored  according 
to  the  function  being  optimized.  This  score  is  then 
used  to  cull  the  population  in  a  way  that  gives 
higher  scoring  individuals  a  greater  chance  of  sur- 
vival. 


After  the  selection  step,  the  surviving  gene  pool 
is  used  to  produce  the  next  generation  by  a  pro- 
cess analogous  to  mating.  Mating  pairs  are  se- 
lected by  either  random  mating  from  the  entire 
population,  some  form  of  inbred  mating,  or  as- 
sortive  mating  in  which  individuals  with  similar 
traits  are  more  likely  to  mate.  The  pairs  are  used 
to  produce  genetic  material  for  the  next  genera- 
tion by  a  process  analogous  to  sexual  reproduc- 
tion. First,  each  individual's  diploid  genome  is 
used  to  produce  a  haploid  by  combining  each  pair 
of  number  strings  into  a  single  string  by  randomly 
choosing  substrings  from  one  or  the  other.  At  this 
point,  randomized  point  mutations  or  transposi- 
tions may  also  be  introduced.  The  two  haploids 
from  each  mating  pair  are  combined  to  produce 
the  genetic  specification  for  each  individual  in  the 
next  generation.  Each  mating  pair  is  used  to  pro- 
duce several  siblings,  according  to  a  distribution 
normalized  to  ensure  a  constant  total  population 
size.  The  entire  process  is  repeated  for  each  gener- 
ation, using  the  gene  pool  produced  by  one  gener- 
ation as  a  specification  for  the  next. 

The  experiments  that  we  have  conducted  have 
simulated  populations  ranging  in  size  from  512  to 
-106  individuals,  with  between  1  and  256  chro- 
mosomes per  individual.  Chromosome  lengths 
have  ranged  from  10  to  128  bits  per  chromo- 
some, mutation  rates  from  0  to  25%  probability  of 
mutation  per  bit  per  generation,  and  crossover 
frequencies  ranged  from  0  to  an  average  of  4  per 
chromosome.  Using  a  Connection  Machine®  #1 
with  65  536  processors,  a  typical  experiment  pro- 
gresses at  about  100  to  1000  generations  per 
minute,  depending  on  population  size  and  on  the 
complexity  of  the  fitness  function. 

2.  Sorting  networks 

As  an  example  of  how  simulated  evolution  can 
be  applied  to  a  complex  optimization  problem,  we 
consider  the  problem  of  finding  minimal  sorting 
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Fig.  1.  Sorting  network. 

networks  for  a  given  number  of  elements.  A  sort- 
ing network  [14]  is  a  sorting  algorithm  in  which 
the  sequence  of  comparisons  and  exchanges  of 
data  take  place  in  a  predetermined  order.  Finding 
good  networks  is  a  problem  of  considerable  prac- 
tical importance,  since  it  bears  directly  on  the 
construction  of  optimal  sorting  programs,  switch- 
ing circuits,  and  routing  algorithms  in  intercon- 
nection networks.  Because  of  this,  the  problem  has 
been  well  studied,  particularly  for  networks  that 
sort  numbers  of  elements  that  are  exact  powers  of 
two. 

Sorting  networks  are  typically  implemented  as 
computer  programs,  but  they  have  a  convenient 
graphical  representation,  as  shown  in  fig.  1.  The 
drawing  contains  n  horizontal  lines,  in  this  case 
16,  corresponding  to  the  n  elements  to  be  sorted. 
The  unsorted  input  is  on  the  left,  and  the  sorted 
output  is  on  the  right.  A  comparison-exchange  of 
the  ilh  and  y'th  elements  is  indicated  by  an  arrow 
from  the  ;'th  to  the  y'th  line.  Two  specified  ele- 
ments are  compared  and  they  are  exchanged  if 
and  only  if  the  element  at  the  head  of  the  arrow  is 
less  than  the  element  at  the  tail;  the  smallest 
element  will  always  end  up  at  the  tail.  The  sorting 
network  pattern  shown  in  fig.  1  is  a  Batcher  sort 
[1],  which  requires  n  \o%~n  -  1  exchanges  to  sort  n 
elements. 

A  useful  property  of  sorting  networks  is  that 
they  are  relatively  easy  to  test.  A  sorting  network 
that  correctly  sorts  all  sequences  of  1  and  0  will 


Fig.  2.  Green's  60-comparison  sorter. 

correctly  sort  any  sequence,  so  it  is  possible  to  test 
an  H-input  sorting  network  exhaustively  with  2" 
tests. 

In  this  section  we  describe  how  simulated  evolu- 
tion is  used  to  search  for  networks  that  require  a 
small  number  of  exchanges  for  a  given  number  of 
inputs.  In  particular,  the  case  n  =  16  is  of  particu- 
lar interest,  and  has  a  long  history  of  successive 
surprises.  In  1962,  Bose  and  Nelson  [2]  showed  a 
general  method  of  sorting  networks  which  re- 
quired 65  exchanges  for  a  network  of  16  inputs. 
They  conjectured  that  this  was  the  best  possible. 
In  1964.  Batcher  [1],  and  independently.  Floyd 
and  Knuth  [6],  discovered  the  network  shown  in 
fig.  1.  which  requires  only  63  exchanges.  Again,  it 
was  thought  by  many  to  be  the  best  possible,  but 
in  1969.  Shapiro  [14]  discovered  a  network  using 
only  62  exchanges.  Later  that  year.  Green  [14] 
discovered  a  60-comparison  sorter,  shown  in  fig.  2, 
which  stands  as  the  best  known.  These  results  are 
summarized  in  table  1.  For  a  lively  and  more 
detailed  account  of  these  developments,  the  reader 
is  referred  to  the  book  by  Knuth  [14,  pp.  227-229]. 

There  are  two  ways  to  cast  the  search  for  mini- 
mal sorting  networks  as  an  optimization  problem. 
The  first  is  to  search  the  space  of  functional  sort- 
ing networks  for  one  of  minimal  length.  The  sec- 
ond is  to  search  the  space  of  short  sequences  of 
comparison/exchanges  for  ones  that  sort  best. 
The  difficulty  with  the  first  approach  is  that  there 
is  no  obvious  way  of  mutating  a  working  sorting 
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Table  1 

Summary  of  number  of  exchanges  required  for  best  known 
sorting  networks  with  16  inputs. 


Best  known  networks 


1962 

Bose  and  Nelson 

65 

1964 

Batcher,  Knuth 

63 

1969 

Shapiro 

62 

1969 

Green 

60 

Networks  found  by  simulated  evolution 

without  parasites  ,  65 

with  parasites  61 


network  into  another  one  that  is  guaranteed  to 
work,  so  almost  all  mutations  and  recombinations 
will  create  a  network  that  is  outside  of  the  search 
space.  It  is  much  easier  in  the  second  approach  to 
produce  mutations  and  variations  of  a  small  pro- 
gram that  stay  within  the  space  of  small  programs. 
Mutation  can  be  implemented  by  changing  the 
position  of  one  of  the  exchanges,  and  recombi- 
nation by  splicing  the  first  part  of  one  sorting 
network  with  the  last  part  of  another.  This  is 
essentially  the  approach  we  have  adopted. 

One  difficulty  with  this  approach  is  that  even  if 
the  solution  is  in  the  space  of  small  networks,  the 
easiest  paths  to  the  solution  may  not  be.  It  may  be 
easier,  for  example,  to  produce  a  short  correct 
network  by  optimizing  a  slightly  longer  correct 
network  than  by  fixing  a  bug  in  a  short  uncor- 
rect  network.  For  this  reason,  we  have  taken 
advantage  of  the  diploid  representation  of  a  geno- 
type to  allow  longer  networks  to  be  generated  as 
intermediate  solutions. 

The  genotype  of  each  individual  consists  of  15 
pairs  of  chromosomes,  each  consisting  of  8  codons. 
representing  the  digits  of  4  chromosome  pairs. 
Each  codon  is  a  4-bit  number,  representing  an 
index  into  the  elements,  so  the  genotype  of  an 
individual  is  represented  as  30  strings  of  32  bits 
each.  The  phenotype  of  each  individual  (an  in- 
stance of  a  sorting  network)  is  represented  as  an 
ordered  sequence  of  ordered  pairs  of  integers. 
There  is  one  pair  for  each  exchange  within  the 
network.  The  elements  of  the  pair  indicate  which 


elements  are  to  be  compared  and  optionally  ex- 
changed. Each  individual  has  between  60  and  120 
pairs  in  its  phenotype,  corresponding  to  sorting 
networks  with  60  to  120  exchanges. 

The  phenotype  is  generated  from  the  genotype 
by  traversing  the  chromosomes  of  the  genotype  in 
fixed  order,  reading  off  the  pairs  to  appear  in  the 
phenotype.  If  a  pair  of  chromosomes  is  homozy- 
gous at  a  given  position  (if  the  same  pair  is  speci- 
fied in  both  chromosomes),  then  only  a  single  pair 
is  generated  in  the  phenotype.  If  the  site  is  het- 
erozygous, then  both  pairs  are  generated.  Thus  the 
phenotype  will  contain  between  60  and  120  ex- 
changes, depending  on  the  heterozygosity  of  the 
genotype.  Sixty  was  chosen  as  the  minimum  size 
so  that  a  completely  homozygous  genotype  would 
produce  a  sorting  network  that  matches  the  best 
known  solution.  Because  most  of  the  known  mini- 
mal 16-input  networks  begin  with  the  same  pat- 
tern of  32  exchanges,  the  gene  pool  is  initialized  to 
be  homozygous  for  these  exchanges.  The  rest  of 
the  sites  are  initialized  randomly. 

Once  a  phenotype  is  produced,  it  is  scored 
according  to  how  well  it  sorts.  One  measure  of 
ability  to  sort  is  the  percentage  of  input  cases  for 
which  the  network  produces  the  correct  output. 
This  measure  is  convenient  for  two  reasons.  First, 
it  offers  partial  credit  for  partial  solutions.  Second, 
it  can  be  conveniently  approximated  by  trying  out 
the  network  on  a  random  sample  of  test  cases. 
After  scoring,  the  population  is  culled  by  trunca- 
tion selection  at  the  50%  level;  only  the  best 
scoring  half  of  the  population  is  allowed  to  con- 
tribute to  the  gene  pool  of  the  next  generation. 

To  implement  recombination,  the  gamete  pool 
is  generated  by  crossover  among  pairs  of  chromo- 
somes. For  each  chromosome  pair  in  the  surviving 
population,  a  crossover  point  is  randomly  and 
independently  chosen,  and  a  haploid  gamete  is 
produced  by  taking  the  codons  before  the  cross- 
over point  from  the  first  member  of  each  chromo- 
some pair,  and  the  codons  after  the  crossover 
point  from  the  second  member.  Thus,  there  is 
exactly  one  crossover  per  chromosome  pair  per 
generation.  Point  mutations  are  then  introduced  in 
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the  gamete  pool  at  a  rate  of  one  mutation  per  one 
thousand  sites  per  generation. 

The  next  stage  is  the  selection  of  mates.  One 
way  to  do  this  would  be  to  choose  pairs  randomly, 
but  our  experience  suggests  that  it  is  better  to  use 
a  mating  program  with  some  type  of  spatial  local- 
ity. This  increases  the  average  inbreeding  coeffi- 
cient and  allows  the  population  to  divide  into 
locally  mating  demes.  The  sorting  networks  evolve 
on  a  two-dimensional  grid  with  torroidal  bound- 
ary conditions.  Mating  pairs  are  chosen  to  be 
nearby  in  the  grid.  Specifically,  the  x  and  v  dis- 
placement of  an  individual  from  its  mate  is  a 
binomial  approximation  of  a  Gaussian  distribu- 
tion. Mating  consists  of  the  exchange  of  haploid 
gametes.  After  a  pair  mates,  they  are  replaced  by 
their  offspring  in  the  same  spatial  location,  so  the 
genetic  material  remains  spatially  local. 

Simulations  were  performed  using  the  proce- 
dure on  populations  of  65  536  individuals  for  up 
to  5000  generations.  Typically,  one  solution,  or  a 
few  equal  scoring  solutions,  were  discovered  rela- 
tively early  in  the  run.  These  solutions  and  their 
variants  then  spread  until  they  accounted  for  most 
of  the  genetic  material  in  the  population.  In  cases 
where  there  was  more  than  one  equally  good  solu- 
tion, each  "species"  dominated  one  area  of  the 
grid.  The  areas  were  separated  by  a  boundary 
layer  of  non-viable  crosses.  Once  these  boundaries 
were  established,  the  population  would  usually 
make  no  further  progress.  The  successful  networks 
tend  to  be  short  because  the  descendant  of  het- 
erozygotes  tended  to  be  missing  crucial  exchanges 
(recessive  lethals).  The  best  sorting  networks  found 
by  this  procedure  contained  65  exchanges. 

3.  The  co-evolution  of  parasites 

While  the  evolution  of  the  sorting  networks 
produced  respectable  results,  it  was  evident  on 
detailed  examination  of  the  runs  that  a  great  deal 
of  computation  was  being  wasted.  There  were  two 
major  sources  of  inefficiency.  One  was  a  classical 
problem  of  local  optima:  once  the  system  found  a 


reasonable  solution,  it  was  difficult  to  make 
progress  without  temporarily  making  things  worse. 
The  second  problem  was  an  inefficiency  in  the 
testing  process.  After  the  first  few  generations, 
most  of  the  tests  performed  were  sorted  success- 
fully by  almost  all  viable  networks,  so  they  pro- 
vided little  information  about  differential  fitness. 
Many  of  the  tests  were  too  "easy."  Unfortunately, 
the  discriminative  value  of  a  test  depends  on  the 
solutions  that  initially  evolve,  and  in  the  case 
where  several  solutions  evolve,  the  value  of  a  given 
test  varies  from  one  sub-population  to  another. 

To  overcome  these  two  difficulties,  various 
methods  were  implemented  for  accelerating 
progress  by  encouraging  a  wider  diversity  of  solu- 
tions and  limiting  the  number  of  redundant  test 
cases.  Three  general  methods  were  investigated: 
varying  the  test  cases  over  time,  varying  the  test 
cases  spatially,  and  varying  the  test  cases  automat- 
ically by  independent  evolution.  Because  the  third 
case  has  yielded  the  most  interesting  results,  only 
it  will  be  described  in  detail. 

The  co-evolution  of  test  cases  is  analogous  to 
the  biological  evolution  of  a  host  parasite,  or  of 
prey  and  predator.  Hamilton  has  used  both  com- 
puter simulation  and  mathematical /biological  ar- 
guments to  show  that  such  co-evolution  can  be  a 
generator  of  genetic  diversity  [7-11].  The  im- 
proved optimization  procedure  uses  this  idea  to 
increase  the  efficiency  of  the  search. 

In  the  improved  procedure,  there  are  two  inde- 
pendent gene  pools,  each  evolving  according  to 
the  selection/mutation/recombination  sequence 
outlined  above.  One  population,  the  "  hosts",  rep- 
resents sorting  networks,  while  the  other  popula- 
tion, the  "  parasites",  represents  test  cases.  (These 
two  populations  might  also  be  considered  as 
"prey"  and  "predator",  since  their  evolution  rates 
are  comparable.)  Both  populations  evolve  on  the 
same  grid,  and  their  interaction  is  through  their 
fitness  functions.  The  sorting  networks  are  scored 
according  to  the  test  cases  provided  by  the  para- 
sites at  the  same  grid  location.  The  parasites  are 
scored  according  to  how  well  they  find  flaws  in 
sorting  networks.  Specifically,  the  phenotype  of 
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Fig.  3.  61  exchanges. 


each  parasite  is  a  group  of  10  to  20  test  cases,  and 
its  score  is  the  number  of  these  tests  that  the 
corresponding  sorting  network  fails  to  pass.  The 
fitness  functions  of  the  host  sorting  networks  and 
the  parasitic  sets  of  test  patterns  are  complemen- 
tary in  the  sense  that  a  success  of  the  sorting 
network  represents  a  failure  of  the  test  pattern 
and  vice  versa. 

The  benefits  of  allowing  the  test  cases  to  co- 
evolve  are  twofold.  First,  it  helps  prevent  large 
portions  of  the  population  from  becoming  stuck  in 
local  optima.  As  soon  as  a  large  but  imperfect 
sub-population  evolves,  it  becomes  an  attractive 
target  toward  which  the  parasitic  test  cases  are 
likely  to  evolve.  The  co-evolving  test  cases  imple- 
ment a  frequency  selective  fitness  function  for  the 
sorting  networks  that  discourages  large  numbers 
of  individuals  from  adopting  the  same  non-opti- 
mal strategy.  Successive  waves  of  epidemic  and 
immunity  keep  the  population  in  a  constant  state 
of  flux.  While  systems  with  a  fixed  fitness  criteria 
tended  to  get  stuck  in  a  few  non-optimal  states 
after  a  few  hundred  generations,  runs  with  co- 
evolving  test  cases  showed  no  such  tendency,  even 
after  tens  of  thousands  of  generations. 

The  second  advantage  of  co-evolving  the  para- 
sites is  that  testing  becomes  more  efficient.  Since 
only  test-case  sets  that  show  up  weaknesses  are 
widely  represented  in  the  population,  it  is  suffi- 
cient to  apply  only  a  few  tests  to  an  individual 


each  generation.  Thus,  the  computation  time  per 
generation  is  significantly  less.  These  two  factors 
taken  together  make  it  both  more  practical  and 
more  productive  to  allow  the  system  to  run  for 
larger  numbers  of  generations. 

The  runs  with  co-evolving  parasites  produced 
consistently  better  and  faster  results  than  those 
without.  Fig.  3  shows  the  best  result  found  to  date, 
which  requires  61  exchange  elements.  This  is  an 
improvement  over  Batcher's  and  Shapiro's  solu- 
tions, and  over  the  results  of  the  simulation  with- 
out parasites.  It  is  still  not  the  optimum  network, 
since  it  requires  one  more  sorting  exchange  than 
the  construction  of  Green. 

These  preliminary  results  are  encouraging.  They 
demonstrate  that  simulated  evolution  of  co-evolv- 
ing parasites  is  a  useful  procedure  for  finding  good 
solutions  to  a  complex  optimization  problem.  We 
are  currently  applying  similar  techniques  to  other 
applications  in  an  attempt  to  understand  the  range 
of  applicability.  It  is  ironic,  but  perhaps  not  sur- 
prising, that  our  attempts  to  improve  simulated 
evolution  as  an  optimization  procedure  continue 
to  take  us  closer  to  real  biological  systems. 
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